Skip to content

feat: Support removing properties from catalogs #713

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Mar 14, 2025
Merged

feat: Support removing properties from catalogs #713

merged 10 commits into from
Mar 14, 2025

Conversation

sbernauer
Copy link
Member

@sbernauer sbernauer commented Feb 28, 2025 •
edited by maltesander
Loading

Description

Part of #712.
I was also running into this in the past, so a nice quality of life feature.

Adds a new field, see added docs for details:

apiVersion: trino.stackable.tech/v1alpha1
kind: TrinoCatalog
spec:
  connector:
    hive: {}
  configOverrides:
    hive.metastore.username: trino
  experimentalConfigRemovals: # <<< New field
    - property.which.blocks.trino.startup

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes
# Author
- [x] Changes are OpenShift compatible
- [x] CRD changes approved
- [x] CRD documentation for all fields, following the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs/style-guide).
- [ ] Helm chart can be installed and deployed operator works
- [ ] Integration tests passed (for non trivial changes)
- [x] Changes need to be "offline" compatible

# Reviewer
- [x] Code contains useful comments
- [ ] Code contains useful logging statements
- [ ] (Integration-)Test cases added
- [x] Documentation added or updated. Follows the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs/style-guide).
- [x] Changelog updated
- [x] Cargo.toml only contains references to git tags (not specific commits or branches)

# Acceptance
- [ ] Feature Tracker has been updated
- [ ] Proper release label has been added
- [ ] [Roadmap](https://github.com/orgs/stackabletech/projects/25/views/1) has been updated

@sbernauer sbernauer self-assigned this Feb 28, 2025
@sbernauer sbernauer mentioned this pull request Feb 28, 2025
Open
@NickLarsenNZ
Copy link
Member

Is there any known negative consequence to doing this?

@sbernauer
Copy link
Member Author

Is there any known negative consequence to doing this?

I'm not aware of any. Trino is a bit special than other tools as it refuses to start up for any unneeded property

@nightkr
Copy link
Member

nightkr commented Mar 3, 2025

It feels weird to have this "seem" so unrelated to configOverrides.. how about something like:

apiVersion: trino.stackable.tech/v1alpha1
kind: TrinoCatalog
spec:
  connector:
    hive: {}
  configOverrides:
    hive.metastore.username: trino
    property.which.blocks.trino.startup: null # <<< removes field

Another option would be to follow your proposal for v1alpha1, but switch to something like this in v1alpha2:

apiVersion: trino.stackable.tech/v1alpha2
kind: TrinoCatalog
spec:
  connector:
    hive: {}
  configOverrides:
    set: # <<< was configOverrides
      hive.metastore.username: trino
    remove: # <<< was experimentalConfigRemovals
      - property.which.blocks.trino.startup

@sbernauer
Copy link
Member Author

Yeah, exactly

This field is experimental, as ideally some general solution for the removal of properties is found and added to configOverrides. This mechanism would replace this field.

IMHO we need this solution for all operators, which is a big task I don't want to do right now.
The current proposal is a quick temporary solution until we figured out the "correct" way of doing it.
I was also thinking of null being special, however that would cause trino-op to have a different behavior as all other operators, which is also not great.

@sbernauer
Copy link
Member Author

Moving this to Voting, please vote on this comment

@sbernauer
Copy link
Member Author

Decision accepted in planning

@sbernauer sbernauer moved this to Development: Waiting for Review in Stackable Engineering Mar 12, 2025
@sbernauer sbernauer closed this Mar 13, 2025
@sbernauer sbernauer reopened this Mar 13, 2025
@maltesander maltesander self-requested a review March 13, 2025 08:57
@maltesander maltesander moved this from Development: Waiting for Review to Development: In Review in Stackable Engineering Mar 13, 2025
maltesander
maltesander requested changes Mar 13, 2025
View reviewed changes
Copy link
Member

@maltesander maltesander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the worker fail their probes

  Warning  Unhealthy          4m45s (x5 over 28m)   kubelet            Readiness probe failed: Get "https://10.244.0.22:8443/v1/info": dial tcp 10.244.0.22:8443: connect: connection refused                                                                              
  Warning  Unhealthy          4m45s (x2 over 28m)   kubelet            Liveness probe failed: Get "https://10.244.0.22:8443/v1/info": unexpected EOF                                                                                                                       
  Warning  Unhealthy          4m45s                 kubelet            Readiness probe failed: Get "https://10.244.0.22:8443/v1/info": unexpected EOF  
``

maltesander
maltesander approved these changes Mar 13, 2025
View reviewed changes
Copy link
Member

@maltesander maltesander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@sbernauer sbernauer added this pull request to the merge queue Mar 14, 2025
@sbernauer sbernauer moved this from Development: In Review to Development: Done in Stackable Engineering Mar 14, 2025
Merged via the queue into main with commit 931a8be Mar 14, 2025
16 of 17 checks passed
@sbernauer sbernauer deleted the feat/support-catalog-property-removal branch March 14, 2025 08:03
@lfrancke
Copy link
Member

Please add a link to the docs and a release note snippet.

@lfrancke lfrancke moved this from Development: Done to Acceptance: In Progress in Stackable Engineering Mar 26, 2025
@sbernauer
Copy link
Member Author

Docs: https://docs.stackable.tech/home/nightly/trino/usage-guide/catalogs/#_config_overrides_and_config_removals
Release notes (I don't think it's important enough for the SDP release notes, but let others decide):

trino-operator/CHANGELOG.md

Lines 15 to 16 in c3a86db

- Support removing properties from catalogs.
This is helpful, because Trino fails to start in case you have any unused config properties ([#713]).

@lfrancke lfrancke moved this from Acceptance: In Progress to Done in Stackable Engineering Mar 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@maltesander maltesander maltesander approved these changes

Assignees

@sbernauer sbernauer

Labels
customer-request release/25.3.0
Projects
Stackable Engineering
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@sbernauer @NickLarsenNZ @nightkr @lfrancke @maltesander